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Abstract — In 1995, Best et al. published a formula for the exact 
bit error probability for Viterbi decoding of the rate R = 1/2, 
memory m — 1 (2-state) convolutional encoder with generator 
matrix G(D) = (1 1 + D) when used to communicate over 
the binary symmetric channel. Their formula was later extended 
to the rate R = 1/2, memory m = 2 (4-state) convolutional 
encoder with generator matrix G(D) = (1 + D 2 1 + D + D 2 ) 
by Lentmaier et al. 

In this paper, a different approach to derive the exact bit error 
probability is described. A general recurrent matrix equation, 
connecting the average information weight at the current and 
previous states of a trellis section of the Viterbi decoder, is derived 
and solved. The general solution of this matrix equation yields 
a closed form expression for the exact bit error probability. As 
special cases, the expressions obtained by Best et al. for the 
2-state encoder and by Lentmaier et al. for a 4-state encoder 
are obtained. The closed form expression derived in this paper 
is evaluated for various realizations of encoders, including rate 
R— 1/2 and R = 2/3 encoders, of as many as 16 states. 

Moreover, it is shown that it is straightforward to extend the 
approach to communication over the quantized additive white 
Gaussian noise channel. 

Index Terms — additive white Gaussian noise channel, binary 
symmetric channel, bit error probability, convolutional code, con- 
volutional encoder, exact bit error probability, Viterbi decoding 



I. Introduction 

In 1971, Viterbi (Tl published a nowadays classical upper 
bound on the bit error probability for Viterbi decoding, 
when convolutional codes are used to communicate over the 
binary symmetric channel (BSC). This bound was derived 
from the extended path weight enumerators, obtained using a 
signal flow chart technique for convolutional encoders. Later, 
van de Meeberg [2] used a very clever observation to tighten 
Viterbi's bound for large signal-to-noise ratios (SNRs). 

The challenging problem of deriving an expression for the 
exact (decoding) bit error probability was first addressed by 
Morrissey in 1970 pj for a suboptimal feedback decoding 
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algorithm. He obtained the same expression for the exact bit 
error probability for the rate R = 1/2, memory m = 1 (2- 
state) convolutional encoder with generator matrix G(D) = 
(1 1 +D) that Best et al. p] obtained for Viterbi decoding. 
Their method is based on considering a Markov chain of the 
so-called metric states of the Viterbi decoder; an approach due 
to Burnashev and Cohn [5|. An extension of this method to the 
rate R — 1/2 memory m — 2 (4-state) convolutional encoder 



D z 1 + D + D z 



was 



with generator matrix G(D) = (1 
published by Lentmaier et al. (6j. 

In this paper we use a different and more general approach 
to derive a closed form expression for the exact (decoding) 
bit error probability for Viterbi decoding of convolutional 
encoders, when communicating over the BSC as well as the 
quantized additive white Gaussian noise (AWGN) channel. 
Our new method allows the calculation of the exact bit error 
probability for more complex encoders in a wider range of 
code rates than the methods of Q and J6}. By considering 
a random tie-breaking strategy, we average the information 
weights over the channel noise sequence and the sequence 
of random decisions based on coin-flippings (where the coin 
may have more than two sides depending on the code rate). 
Unlike the backward recursion in [4| and |6|, the bit error 
probability averaged over time is obtained by deriving and 
solving a recurrent matrix equation for the average information 
weights at the current and previous states of a trellis section 
when the maximum-likelihood branches are decided by the 
Viterbi decoder at the current step. 

To illustrate our method, we use a rate R = 2/3 systematic 
convolutional 2-state encoder whose minimal realization is 
given in observer canonical form, since this encoder is both 
general and simple. 

In Section [IlJ the problem of computing the exact bit 
error probability is reformulated via the average information 
weights. A recurrent matrix equation for these average in- 



formation weights is derived in Section III and solved in 
Section [IV] In Section [V] we give additional examples of 
rate R — 1/2 and R = 2/3 encoders of various memories. 
Furthermore, we analyze a rate R = 1/2 4-state encoder used 
to communicate over the quantized additive white Gaussian 
noise (AWGN) channel and show an interesting result that 
would be difficult to obtain without being able to calculate 
the exact bit error probability. 

Before proceeding, we would like to emphasize that the bit 
error probability is an encoder property, neither a generator 
matrix property nor a convolutional code property. 
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II. Problem Formulation via the Average 
Information Weights 

Assume that the all-zero sequence is transmitted over a 
BSC with crossover probability p and let Wt{cr) denote the 
weight of the information sequence corresponding to the code 
sequence decided by the Viterbi decoder at state a and time in- 
stant t. If the initial values Wo(a) are known, then the random 
process Wt(cr), t = 0, 1, 2, . . ., is a function of the random 
sequence of the received c-tuples r T , t = 0, 1, . . . , t — 1, and 
the coin-dippings used to resolve ties. 

Our goal is to determine the mathematical expectation of the 
random variable Wt {a) over this ensemble, since for rate R = 
b/c minimal convolutional encoders the bit error probability 
can be computed as the limit 

E[W t (a = 0)} 



Pb 



lim 

t—^-oo 



tb 



(1) 



assuming that this limit exists. 

Remark. If we consider nonminimal encoders, all states 
equivalent to the all-zero state have to be also taken into 
account. 

We consider encoder realizations in both controller and 
observer canonical form and denote the encoder states by cr, 
a G {0, 1, ... , \£\ — 1}, where is the set of all possible 
encoder states. 

During the decoding step at time instant t + 1 the Viterbi 
algorithm computes the cumulative Viterbi branch metric 
vector fi t+1 = (m*+i(0) . . ./x t+ i(|E| - 1)) for the 

time instant t + 1 using the vector fi t and the received c-tuple 
r t . It is convenient to normalize the metrics such that the 
cumulative metrics at every all-zero state will be zero, that is, 
we subtract the value /it(0) from /it(l), A*t(2), ■ ■ ■ , Mt(|E| — 1) 
and introduce the normalized cumulative branch metric vector 

(&(1) <M2)...«M|£|-1)) 
= ( Mt (l)-Mt(0) fH(2)-(H(fl) . . . /xt(|E| - l)-^(0)) 
For example, for a 2-state encoder we obtain the scalar 

<k = Mi) 

while for a 4-state encoder we have the vector 



<t>i 



0t(l) <M2) <M3) 



The elements of the random vector <p t belong to a set 
whose cardinality M depends on the channel model, encoder 
structure, and the tie-breaking rule. Enumerating the vectors 
4> t by numbers <fi t which are random variables taking on M 
different integer values 4>^\ . ■ . , (f)^^ 1 ', the sequence 
of numbers <p t forms an A/-state Markov chain $ t with 

where 



transition probability matrix $ 

4> jk = Pr (q 



<H+1 



= {4>jk) 



<f> t = 



(2) 



Let Wt be the vector of information weights at time instant 
t that depends both on the \ J2\ encoder states a t and on the M 
normalized cumulative metrics </> t ; that is, Wt is expressed as 
the following vector with M entries 



W t =(Wt(<r = 0) W t (cr=l) ... W t (cr 



(3) 



,.(2) 




v (2) 
„(3) 



Fig. 1: A minimal encoder for the generator matrix given in 
equation ([6j. 



where 

Wt(a)=(w t (cj>W,a) W t {4> {1 \a) ... Wt^" 1 ), a)) (4) 
Then ([TJ can be rewritten as 

P h = lim E M^M = Um EtL-oE[W t (^\- = 0)} 
t->oo tb t->oo tb 



= lim 

t— >oo 



E[W t (a = 0)]ll M w t (a = 0)ll M 



tb 



= lim 

t— > oo 



tb 



W t 



= lim -^(1i,m0i )M • 

t-voo tb 



0i,m) 7 



(5) 



where 1i,m and Oi m denote the all-one and the all-zero row 
vectors of length M, respectively, w t represents the length 
M|E| vector of the average information weights, while the 
length M vector of average information weights at the state 
a is given by w t (a). Note that the mathematical expectations 
in |5]l are computed over the sequences of channel noises and 
coin-flipping decisions. 

To illustrate the introduced notations, we use the rate R = 
2/3 memory in = 1, overall constraint length v = 2, minimal 
encoder with systematic generator matrix 



G(D) 



1 1 + D 
1 1 + D 



(6) 



It has a 2-state realization in observer canonical form as shown 
in Fig. [1] 

Assuming that the normalized cumulative metric state is 
4>t — 0, we obtain the eight trellis sections given in Fig. [2] 
These trellis sections yield the normalized cumulative metric 
states {—1,0,1}. Using <p t — —1 and <f> t — 1, we obtain 
16 additional trellis sections and two additional normalized 
cumulative metric states {—2,2}. From the metrics <fi t = —2 
and 4> t = 2, we get another 16 trellis sections but those will 
not yield any new metrics. Thus, in total we have M = 5 
normalized cumulative metric states <pt € {—2,-1,0,1,2}. 
Together with the eight different received triples, r t — 000, 
001, 010, 100, 011, 101, 110, 111, they correspond to in 
total 40 different trellis sections. The bold branches in Fig. [2] 
correspond to the branches decided by the Viterbi decoder at 
time instant t + 1. When we have more than one branch with 
maximum normalized cumulative metric entering the same 
state, we have a tie which we, in our analysis, resolve by 
fair coin-flipping. 

Hence, the normalized cumulative metric <E> t is a 5-state 
Markov chain with transition probability matrix $ = (4>jk), 
1 < j, k < 5. 



3 




2pq- 



p + pq 




q 3 +p 2 q 

Fig. 3: Illustration of the 5-state Markov chain formed by the 
sequences of normalized cumulative metric states <j> t . 

From the four trellis sections, (a), (b), (g), and (h), in Fig. [2] 
we obtain 

0o(-i) = Pr (r t = 000) + Pr (r t = 001) 



+ Pr(r t = 110) +Pr(r t = 111) 
q 3 + pq 2 + p 2 q + p 3 = p 2 + q 2 



(7) 



while the four trellis sections, (c), (d), (e), and (f), yield 

001 : 

where q = 1 — p. 



01 =PQ 2 +PQ 2 +p 2 q + p 2 q = 2pq (8) 



Similarly, we can obtain the remaining transition probabil- 
ities from the 32 trellis sections not included in Fig. [2] Their 
transition probability matrix follows as 



$ = 



whose metric state Markov chain is shown in Fig. [3] 

Let p t denote the probabilities of the M different nor- 
malized cumulative metric values of $ t , that is, <\> t 6 
{4>(°\ <p^\ ■ ■ ■ , 0^ M ~ 1 - ) }. Their stationary distribution is de- 
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f 3pq 2 





2p 2 q\ 
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p s. 


±3pq 2 
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(9) 
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+ pq 2 
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q 3 - 


f 3p 2 q 





2p 2 q) 





noted = (p^' p^' . . .p^, 1 1 ') and is determined as the 



5 (0) (i) 

->oo Poo 

solution of, for example, the first M 

Pco* =Poo 

and 



1 equations of 



M-l 

E 

i=0 



p 



(10) 



(ID 



For the 2-state convolutional encoder with generator matrix 
(|6]l we obtain 

T = 1 

P °° 1 - p + 10p 2 - 20p 3 + 20p 4 - 8p 5 
/ 1 + 7p - 28p 2 + 66p 3 - 100p 4 + 96p 5 - 56p 6 + 16p 7 \ 

- 3p + 16p 2 - 46p 3 + 80p 4 - 88p 5 + 56p 6 - 16p 7 

- 3p + 10p 2 - 20p 3 + 20p 4 - 8p 5 
- 6p 2 + 26p 3 - 60p 4 + 80p 5 - 56p 6 - 16p 7 

\ - 2p 2 - 6p 3 + 40p 4 - 72p 5 + 56p 6 - 16p 7 J 

In order to compute the exact bit error probability according 
to (I3J, it is necessary to determine w t (a = 0). In the next 
section we will derive a recurrent matrix equation for the 
average information weights and illustrate how to obtain its 
components using as an example the rate R — 2/3 memory 
m = 1 minimal encoder determined by Q. 
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III. Computing the Vector of Average 
Information Weights 

The vector w t describes the dynamics of the information 
weights when we proceed along the trellis and satisfies the 
recurrent equation 



w t+1 = w t A + b t B 
bt+i = b t U 



(12) 



where A and B are M|£| x M |£| nonnegative matrices, IT 
is an M|£| x M|E| stochastic matrix, and |£| = 2 m . Both 
matrices consist of |£| x 
M x M, respectively, where the former satisfy 



|£| submatrices Aij and By of size 



|E|-1 

£ 

i=0 



A„=$-l, j = 0,l,. 



(13) 



since we consider only encoders for which every encoder state 
is reachable with probability 1. 

The matrix A represents the linear part of the affine 
transformation of the information weights while the matrix 
B describes their increments. The submatrices Ay and -By 
describe the updating of the average information weights if the 
transition from state i to state j exists; and are zero otherwise. 
Moreover, the vector b t of length M|£| is the concatenation 
of \J2\ stochastic vectors p t , and hence the M|£| x M|£| 
matrix IT follows as 



n = 



/ 4> 
$ 



V 



\ 





(14) 



For simplicity, we choose the initial value of the vector of the 
information weights to be 



w = 



(15) 



Continuing the previous example, we will illustrate how the 
10 x 10 matrices A and B can be obtained directly from all 40 
trellis sections. For example, the eight trellis sections in Fig. [2] 
determine all transitions from <fi t = to either <pt+i — — 1 or 
<t>t+i = 1. 

To be more specific, consider all transitions from a t = and 
(j>t = to <7t+i = and <frt+i = — 1> as shown in Fig. [2ja), (b), 
(g), and (h). Only Fig. |2|a) and (g) have transitions decided 
by the Viterbi algorithm, which are v t = 000 in Fig. |2|a) and 
v t = 110 in Fig. |2|g), and thus the entry a t = 0, 4>t = 0, 
&t+i = 0, 4>t+i = — 1 in matrix A follows as 



Pr (r t = 000) 
and in matrix B as 



■Pr(r t = 110) = q 3 



-p 2 q 



P (000) Pr (rt = 000) + /3 (110) Pr (r t = 110) 
= + 2p 2 q = 2p 2 q 

where p (v t ) denotes the number of information Is corre- 
sponding to Vt, Since we use coin-flipping to resolve ties, 



we obtain that the entry a t = 0, 4>t 
(Fig. |2|c) and (d)) in matrix A is 



0, a t+ i = 0, 



i = 1 



1 



Pr (r t = 010) + -Pr (r t = 010) 
--Pr (r t = 100) + ^Pr (r t = 100) 



2 P<1 



2 M 



2 P<1 



2 PQ 



2pq 2 



and in matrix B 
1 



P (000) Pr (r t = 010) + -/? (110) Pr (r t = 010) 



1 



/3(000)Pr(r t = 100) 



= 2-°- 



2pq 2 



/3(110)Pr(r t = 100) 
1 



Similarly the entry at = 1 
(Fig. |2|b) and (h)) in matrix A is 



1 

2 " ' 2 

(f>t = 0, (T t+ i = 



2pq 2 = 2pq 2 



= -1 



and in matrix B 



pq +p 



+ 2p A = 2p s 



Finally, the entry at = 1, 4>t = 0, <7t+i = 0^ ^t+i = 1 
(Fig. |2|e) and (f)) in matrix A is given by 



1 2 . 1 



p 2 q 



1 2 i 1 2 n 2 



2p 2 q = 2p 2 q 



2 1 " T 
and in matrix B by 

1 1 2 1 1 
2-0 + -.2^+-.0+- 

The trellis sections in Fig. [2] determine also the entries for 
the transitions a t = 0, <f> t = 0, a t +i — 1. 4>t+x = — 1 an d 
a t = 0, (j>t = 0, (Tt+i = 1, 4>t+i = 1 as well as the transitions 

a t = 1, <jh = 0, a t+1 = 1, <pt+i = -1 and a t = 1, <$> t = 0, 

= 1, <pt+i = !• 
The remaining transitions with t = are never decided 
by the Viterbi algorithm, and hence the corresponding entries 
are zero. The eight trellis sections in Fig. [2] yield 20 entries in 
the matrices A and B, while the 32 trellis sections not shown 
in Fig. [2] yield the remaining 80 entries. For the convolutional 
encoder shown in Fig. [T] we obtain 

A)0 Aqi 

A w An 

where 



A = 



(16) 



.4 



oo 



An = 
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' q 3 +p 2 q 
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-3pq 2 
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V\p 2 q 
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2p 2 g N| 
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p 2 g 














p 3 +pq 2 





2p 2 q 





1 




p 3 


+2p^ 2 





2p 2 q 
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V o 











o / 



(17) 



(18) 



-2 




~3^_2 



-1 




p*+pq z 



p 3 +pq 
V p 3 +pq 2 



o 



\p 3 + \pq 2 




\q 3 + \p 2 q 



2 

\ 



P 2 <J 







3p J q+q 6 



l 




2p 2 g 
pq 2 
2pg 2 / 



(19) 



-2 





\ 

pq 2 

p 2 q+q 3 2pg 2 

|W 2 + 5P 3 2p 2 q+q 3 2 M 2 

\ p 3 +pg 2 3p 2 q+q 3 2pg 2 / 



(20) 



B = 



Boo Boi 
Bio Bn 



(21) 
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-1 
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-2 


/ 2p 2 q 





p 3 + 2pq 2 


2p 2 q^ 


-1 








p 2 g 


pq 2 
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2p 2 q 


2 M 2 
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2p 2 q 





p 3 + 2pq 2 


p 2 g 
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V o 








o / 



(22) 
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4p 2 g 2pq 2 ) 
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-1 1 


2 


/ 
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-h 3 


pq 2 












p 2 q+q 3 2p<j 
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-hp 3 


2p 2 q + q 3 
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p 3 +pq 2 


3p 2 g+<? 3 


2pq 2 J 



(23) 



(24) 



(25) 



IV. Solving the recurrent equation 

Consider the second equation in ( |12) , It follows from ([5]) 
that we are only interested in the asymptotic values, and hence 
letting t tend to infinity yields 



boo = booll 
where can be chosen as 

b oc = (Poo Poo ■ • • Po 



(26) 



(27) 



To obtain the last equality, we took into account that II is a 
block-diagonal matrix whose diagonal elements are given by 
the transition probability matrix $ which satisfies (|T0j. Based 
on these observations, ( [T2] i can be simplified to 



w t+1 = wtA + b^B 



(28) 



By iterating the recurrent equation ( |28] i and using the initial 
value ( fTB) , the vector of the information weights at time instant 
t + 1 is given by 



n+i 



b^BA + b^BA 1 



(29) 



Taking its limit, it follows that 



lira — y- 

t->oo tb 



lim 

t— f oo 



H+l 



tb 

= b^BA^/b 



lim -V6 0O BA*- j 
t->oo tb ' 



(30) 



where ^4°° denotes the limit of the sequence A 1 when t tends 
to infinity and we used the fact that, if a sequence converges 
to a finite limit, then it is Cesaro-summable to the same limit. 
From ( p~3] > it follows that 



satisfies 



e L = (Poo Poo ■•■Poo) 



e L A = e L 



(31) 



(32) 



and hence is a left eigenvector with eigenvalue A = 1. Due 
to the nonnegativity of A, A = 1 is a maximal eigenvalue of 
A (Corollary 8.1.30 (TJ). Let e R denote the right eigenvector 
corresponding to the eigenvalue A = 1 normalized such that 
e^en = 1. If we remove the allzero rows and corresponding 
columns from the matrix A we obtain an irreducible matrix 
which has a unique maximal eigenvalue A = 1 (Lemma 
8.4.3 (7|)- Hence, it follows (Lemma 8.2.7, statement (i) Q) 
that 

A°° = e R e L (33) 
Combining ([31), and ((33]) yields 

, lim 777 = b ooBe R ( Poo Poo . . .p^/b (34) 

t-Hx> tb 

Following Q, by summing up the first M components of the 
vector (p^ p M . . - Poo) on the right side of ( [34) , we obtain 
the closed form expression for the exact bit error probability 

as 



P b = boo£?e R /6 



(35) 
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Example 1: If we draw all 20 trellis sections for the rate 
R = 1/2, memory m = 1 (2-state) convolutional encoder with 
generator matrix G(D) = (1 1 + D) realized in controller 
canonical form, we obtain the normalized cumulative metric 
states {—2,-1,0,1,2}. Its metric state Markov chain yields 
the stationary probability distribution 



10- 1 

BSC crossover probability p 



10- 



Fig. 4: Exact bit error probability for the rate R = 1/2 minimal 



encoders of memory m = 1 (2-state) G(D) = (1 1 
memory m = 2 (4-state) G(D) = (1 + D 2 1 + D 4 
memory m = 3 (8-state) G(D) = (1 + D 2 + D 3 H 
D 2 + L> 3 ), and memory m = 4 (16-state) G(-D) = (1 - 
L» 4 1 + D + D 2 + D 3 + D*). 



\-D), 
D 2 ), 
D + 
D + 



To summarize, the exact bit error probability P\, for Viterbi 
decoding of a rate R = b/c minimal convolutional encoder, 
when communicating over the BSC, is calculated as follows: 

• Construct the set of metric states and find the stationary 
probability distribution p m . 

• Determine the matrices A and B as in Section [TT] and 
compute the right eigenvector eR normalized according 

t0 (Poo Poo---Poo) e R = L 

• Calculate the exact bit error probability P\, using (j35j. 
For the encoder shown in Fig. [Tj we obtain 

(4p - 2p 2 + 67p 3 - 320p 4 + 818p 5 - 936/ - 884p 7 



ft 



2p z 

+5592/ - 11232/ + 13680p 10 - 11008/ 1 



+5760p 12 
-128/ + 
+1440/ - 

= 2p + 4/ + 

70373 7 
-P 



- 1792p 13 + 256p 14 ) / (2 - 5p + 41p 2 
360/ - 892/ + 1600/ - 1904/ 

- 640/ + 128/°) 
5 , 431 , 125 K 32541 , 



-,p 3 ^rP 4 



32 

67672493 in 
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256 y 
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1675587 o 

/ + 



7590667 



16 



-P 



64 



128 



-P 



(36) 



If we instead realize the minimal generator matrix (|6]l in 
controller canonical form, we obtain a nonminimal (4-state) 
encoder with M — 12 normalized cumulative metric state; 
cf., the Remark after (QJ. Its exact bit error probability is 
slightly worse than that of its minimal realization in observer 
canonical form. 

V. Some Examples 

First we consider some rate R = 1/2, memory m = 1, 2, 3, 
and 4 convolutional encoders; that is, encoders with 2, 4, 8, and 
16 states, realized in controller canonical form. In Fig. [4] we 
plot the exact bit error probability for those four convolutional 
encoders. 
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Based on these 20 trellis sections, the 10 x 10 matrices A and 
B are constructed as 



and 



.4 



B = 



A 00 
A w 

5 ,5 
5 ,5 



All 

Am. 
An 



(38) 



(39) 



where O5 5 denotes the 5x5 all-zero matrix. The normalized 
right eigenvector of A is 








pq 

2 
4pq 



e R 



2 - p + 4/ - 4/ 
(2 + 7p - 12/ + 13/ - 12/ 



4/ 



(40) 



2(2 - p + 4p 2 - 4/ 
1 

Finally, inserting ((37]), (|39]), and ((40]) into ([35) yields the 
following expression for the exact bit error probability 

14/ - 23/ + 16/ + 2/ - 16/ + 8/ 

* b 



(1 + 3/ - 2/)(2 -p + 4/ - 4p 3 ) 



7/ 



511 



- 31/ - 
10165 



64/ 



Q« 6 635 7 
86/ - —p' 



4963 



s 



16 



-P 



(41) 



which coincides with the exact bit error probability formula 
given in H). 

Example 2: For the rate R = 1/2, memory m = 2 (4- 
state) convolutional encoder with generator matrix G(D) = 
(1 + D 2 1 + D + D 2 ) realized in controller canonical form, 
we obtain, for example, the four trellis sections for (f) t — (000) 
shown in Fig. [5] The corresponding metric states at times t + 1 
are t+i = (-10 -1) and & +1 = (101). 

Completing the set of trellis sections yields in total i\/ = 31 
different normalized cumulative metric states, and hence the 
124 x 124 matrices A and B have the following block structure 
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Ht r t = 00 fH+l (H 

-,00 2 







00 




1 



1 




1 



00 1 00 



r t = 10 (H+i \H 
00 1 




1 



00 1 00 



r t = 11 (H+i 
00 




1 



t = (OOO) </> t+1 = (-10-1) t = (OOO) <£ t+1 = (101) t = (OOO) t+1 = (lOl) ^ t = (000) t+1 = (-lO-l) 
Fig. 5: Four different trellis sections of the in total 124 for the G(D) = (1 + D 2 1 + D + D 2 ) generator matrix. 



A 



( A 00 

031,31 
\ 031,31 



031,31 
031,31 

An. 
A 3 i 



A 2 
Al2 
031,31 
031,31 



031,31 
031,31 
A 23 
^33 



and 



B = 



031,31 
031,31 
031,31 
031,31 



A 02 
A12 
031,31 
031,31 



031,31 
031,31 
A 23 
A 33 



\ 



(42) 



(43) 



/ 03i : 3i 
031,31 
031,31 

\ 031,31 O3131 031,31 ^33 J 

Following the method for calculating the exact bit error 



probability described in Section IV we obtain 

3519 „ 14351 „ 1267079 



P h = 44p J + 



8 



~P 



31646405 



~^T P " 
978265739 



64 



P 



512 

3931764263 q 
P 



P 



2048 

48978857681 in 

-p 



(44) 



1024 r 32768 
which coincides with the previously obtained result by Lent- 
maier et al. (5). 

Example 3: For the rate R — 1/2, memory m — 3 (8-state) 
convolutional encoder with generator matrix G(D) = (1 + 
D 2 + D 3 1 + D + D 2 + D 3 ) realized in controller canonical 
form we have M = 433 normalized cumulative metric states 
and the A and B matrices are of size 433 • 2 3 x 433 • 2 3 . 

Since the complexity of the symbolic derivations increases 
greatly, we can only obtain a numerical solution of ( (35| ), as 
shown in Fig. |4] 

Example 4: For the rate R = 1/2, memory rn = 4 (16- 
state) convolutional encoder with generator matrix G(D) = 
(1 + D' 2 + D 3 + D 4 1 + D + L> 4 ) realized in controller 
canonical form, we have as many as M — 188687 normalized 
cumulative metric states. Thus, the matrices A and B are of 
size 188687 • 2 4 x 188687 • 2 4 . The corresponding numerical 
solution of ([35) is plotted in Fig. [4] 

The obvious next step is to try a rate R = 1/2, memory 
m = 5 (32-state) convolutional encoder. We tried the generator 
matrix G{D) = (1 + D + D 2 + D 3 + D* + D 5 1 + D 3 + D 5 ) 
realized in controller canonical form but were only able to 
show that the number of cumulative normalized metric states 
M exceeds 4130000. 



Example 5: Consider the generator matrix G\(D) = (1 + 
D 2 1 + D + D 2 ) and its equivalent systematic generator 
matrices G 2 {D) = (1 (l + D 2 )/(l+D + D 2 )) and G 3 (D) = 
(1 (1 + D + D 2 )/(l + D 2 )). When realized in controller 
canonical form, all three realizations have M = 31 normalized 
cumulative metric states. The exact bit error probability for 
Gi(D) is given by (|44j>. For G 2 (D) and G 3 (D) we obtain 

163 , 365 a 24045 K 1557571 « 



Ph = 



-Y p3 + -T p4 

23008183 7 

H V + 

512 

4249634709 q 



1191386637 



128 



-P 



-p 



and 



P, 



141 



-P 



8192 
1739 



2048 

132555764497 , 

p — 

8192 y 

71899 K 1717003 
128 



(45) 



P 



2635041 
128 

9896230051 q 
-P 



v - 

32 

540374847 „ 

P 8 



P 



1024 

402578056909 , 
p 



(46) 



8192 r 32768 
respectively. The corresponding numerical results are illus 
trated in Fig. [6] 




BSC crossover probability p 

Fig. 6: Exact bit error probability for the rate R — 1/2 memory 
to = 2 minimal encoders with G\{D) = (1 



D 2 ), G 2 (D) = (1 (1 + D 2 )/(1 
(1 (l + D + D 2 )/(l + D 2 )). 



D z 1 + D + 
D + D 2 )), and G 3 (D) = 



8 



10- 1 
io- 2 

10~ 3 
1(T 4 
10~ 5 
10~ 6 
IO- 7 



4 states 
• 8-state 
16-state 



IO" 1 1(T 2 
BSC crossover probability p 



ltr 



Fig. 7: Exact bit error probability for the rate R — 2/3, overall 
constraint length v = 2, 3, and 4 (4-state, 8-state, and 16-state, 
respectively) minimal encoders whose generator matrices are 
given in Table [IJ 

TABLE I: Rate R = 2/3 generator matrices 



G(D) 


#states 


f^free 


M 


( D l + D l+D \ 
\ 1 D l+D ) 


4 


3 


19 


( l + D D 1 \ 
\_ D 2 1 l + D + D 2 J 


8 


4 


347 


( D + D 2 1 l + D 2 \ 
\ 1 D + D 2 l + D + D 2 ) 


16 


5 


15867 



Example 6: The exact bit error probabilities for the rate 
R = 2/3 4-state, 8-state, and 16-state generator matrices, 
given in Table [IJ and realized in controller canonical form, 
are plotted in Fig. [7] 

As an example, the 4-state encoder has the exact bit error 
probability 



Ph = 



67 



17761 



2147069 



1055513863 



-P 



2 1 48 
123829521991 fi 
-V 



-P 



559872 
27081094434882419 8 

P 8 



648 r 46656 
67343848419229 7 
P 



2176782336 
1944829319763332473469 10 

2821109907456 P 



60466176 

477727138796620247 q 
P 



8707129344 



(47) 



If we replace the BSC with the quantized additive white 
Gaussian noise (AWGN) channel, the calculation of the exact 
bit error probability follows the same method as described 
in Section 



IV 



but the computational complexity increases 
dramatically as illustrated by the following example. 

Example 7: Consider the generator matrix G(D) = (1 + 
D 2 1 + D + D 2 ) used to communicate over a quan- 
tized AWGN channel. We use different quantization methods, 
namely, uniform quantization JSJ, (9j and Massey quantization 
(10), (TTJ; see Fig. [9] 




Fig. 8: Exact bit error probability for the rate R = 1/2, 
memory m = 2 (4-state) encoder with G(D) = (1 + D 2 1 + 
D + D 2 ) used to communicate over an AWGN channel with 
different quantization levels. 



Uniform 

-3 -2-10 1 



3 7-Ievels 



-1 1 2 3 4 8-Ievels 
1 1 1 1 



4 9-Ievels 



Massey 

T 2 T 3 T 4 T s 



T 6 



Ti 



T 2 T a T 4 T 5 
— • 1 1 ■— 



T 6 



T 7 



Ti 



T 2 T 3 Ti T 5 T s T r 
1 1 1 . 1 1 1 



7-Ievels 



^-levels 



9-Ievels 



Fig. 9: Examples of uniform and Massey quantizations for an 
AWGN channel with SNR = OdB. 

The uniform intervals were determined by optimizing the 
cut-off rate Rq. The Massey quantization thresholds Ti be- 
tween intervals were also determined by optimizing Rq, but 
allowing for nonuniform intervals. The realization in controller 
canonical form yields that, for all signal to noise ratios (SNRs), 
E\,/No, and uniform quantization with 7, 8, and 9 levels, 
the number of the normalized cumulative metric states is 
M = 1013, M = 2143, and M = 2281, respectively. 
However, for the Massey quantization the number of normal- 
ized cumulative metric states varies with both the number 
of levels and the SNR. Moreover, these numbers are much 
higher. For example, considering the interval between OdB 
and 3.5 dB with 8 quantization levels, we have M = 16639 
for E b /N < 2.43 dB, while for E h /N > 2.43 dB we obtain 
M — 17019. The exact bit error probability for this 4-state 
encoder is plotted for all different quantizations in Fig. [8] 
ordered from worst (top) to best (bottom) as 

(i) Uniform 8 levels 

(ii) Uniform 7 levels 

(iii) Massey 7 levels 

(iv) Uniform 9 levels 

(v) Massey 8 levels 

(vi) Massey 9 levels 
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All differences are very small, and hence it is hard to 
distinguish all the curves. It is interesting to notice that using 
7 instead of 8 uniform quantization levels yields a better bit 
error probability. However, this is not surprising since the 
presence of a quantization bin around zero typically improves 
the quantization performance. Moreover, the number of cu- 
mulative normalized metric states for 7 quantization levels is 
only about one half of that for 8 quantization levels. Notice 
that such a subtle comparison of channel output quantizers has 
only become possible due to the closed form expression for 
the exact bit error probability. 

VI. Conclusion 

We have derived a closed form expression for the exact 
bit error probability for Viterbi decoding of convolutional 
codes using a recurrent matrix equation. In particular, the 
described method is feasible to evaluate the performance of 
encoders with as many as 16 states when communicating 
over the BSC. By applying our new approach to a 4-state 
encoder used to communicate over the quantized AWGN 
channel, the expression for the exact error probability for 
Viterbi decoding is also derived. In particular, it is shown 
that the proposed technique can be used to select the optimal 
encoder implementation as well as the optimal channel output 
quantizer based on comparing their corresponding exact bit 
decoding error probability. 
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